Validation of a camera cleaning system

ABSTRACT

Devices, systems, and methods are provided for testing and validation of a camera. A device may capture a first image of a target using a camera, wherein the camera is in a clean state, and wherein the target is in a line of sight of the camera. The device may apply an obstruction to a portion of a lens of the camera. The device may apply a camera cleaning system to the lens of the camera. The device may capture a post-clean image after applying the camera cleaning system. The device may determine a post-clean SSIM score based on comparing the post clean image to the first image. The device may compare the post-clean SSIM score to a validation threshold. The device may determine a validation state of the camera cleaning system based on the comparison.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/020,297 filed Sep. 14, 2020, the disclosure of which is herebyincorporated in its entirety by reference herein.

TECHNICAL FIELD

This disclosure generally relates to devices, systems, and methods forvalidation of a camera cleaning system.

BACKGROUND

Many vehicles use one or more cameras for various purposes. For example,a camera mounted on a vehicle may be coupled to an image processingdevice that processes images to detect objects in the vicinity of thevehicle. Proper operation of a camera in the presence of some amount ofobstruction is an important aspect of the camera. The camera should notbe interrupted from its normal function under the presence ofobstructions. Some of these obstructions may include debris, mud, rain,bugs, or other obstructions that may hinder the normal operation of thecamera. These obstructions may alter the image quality of the cameraafter getting deposited on the camera lens. Thus, there is a need toenhance the testing and validation of a camera to ensure thatobstructions do not cause inconsistent or unreliable image quality thatundermines the camera's normal operation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary vehicle equipped with multiple cameras,in accordance with one or more example embodiments of the presentdisclosure.

FIG. 2 depicts an illustrative schematic diagram for camera cleaningvalidation, in accordance with one or more example embodiments of thepresent disclosure.

FIG. 3A depicts an illustrative schematic diagram for camera cleaningvalidation, in accordance with one or more example embodiments of thepresent disclosure.

FIG. 3B depicts an illustrative schematic diagram for camera cleaningvalidation, in accordance with one or more example embodiments of thepresent disclosure.

FIG. 3C depicts an illustrative schematic diagram for camera cleaningvalidation, in accordance with one or more example embodiments of thepresent disclosure.

FIG. 4 illustrates a flow diagram of an illustrative process for acamera cleaning validation system, in accordance with one or moreexample embodiments of the present disclosure.

FIG. 5 is a block diagram illustrating an example of a computing deviceor computer system upon which any of one or more techniques (e.g.,methods) may be performed, in accordance with one or more exampleembodiments of the present disclosure.

Certain implementations will now be described more fully below withreference to the accompanying drawings, in which various implementationsand/or aspects are shown. However, various aspects may be implemented inmany different forms and should not be construed as limited to theimplementations set forth herein; rather, these implementations areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the disclosure to those skilled in the art.Like numbers in the figures refer to like elements throughout. Hence, ifa feature is used across several drawings, the number used to identifythe feature in the drawing where the feature first appeared will be usedin later drawings.

DETAILED DESCRIPTION

Example embodiments described herein provide certain systems, methods,and devices for enhanced multi spectral sensor calibration.

In one or more embodiments, a camera cleaning validation system mayfacilitate the setup of a camera in an obstruction environment which isconstrained in both its required setup as well as the environment it isin. A device (e.g., a camera) should not be interrupted from its normalfunction. For example, if an obstruction is deposited on the lens of acamera, a camera cleaning system should be applied in order to attemptto return the camera to its normal function by clearing the obstructionoff of the camera lens to a certain degree.

In one or more embodiments, a camera cleaning validation system maycombine a scale-invariant target with a target-independent camerametric, which allows a wide variety of errors in camera performance tobe detected from an expected baseline without an application-specificbehavior being required. An application-specific behavior may be tied tothe type of tests being performed, such as sharpness test, blurred linestest, resolution test, modulation transfer function (MTF), chromaticaberration test, white balance, color response, color error, and anyother specific tests.

In one or more embodiments, a camera cleaning validation system mayfacilitate a validation test for a camera under test. A camera cleaningvalidation system may provide a mechanism to allow a pass or failcriteria to be judged on the camera under test in real-time during thetesting and provides a target framework and a backend processingframework together in real-time application. A camera cleaningvalidation system may allow for a testing and validation environmentthat does not necessarily require tuning the camera for ideal lightningconditions, setup, distance, etc., which are typically required bycamera tests. That is, it is not necessary to determine pixel sizes ordetermine a specific color palette, which is typically used to performimage quality tests. A controlled environment not required, but rather acamera cleaning validation system may reduce the noise level from theenvironment. In fact, the camera cleaning validation system may allowmore variation in the test setup to achieve a higher level ofrepeatability and reproducibility, which in turn allows higherresolution to differentiate between ‘pass’ and ‘fail’ conditions.

In one or more embodiments, a camera cleaning validation system mayfacilitate an application-independent methodology by using a metricassociated with the validation of a camera system. That is, the abilityto measure the camera's sensitivity to the obstruction that is applied(regardless of actual cleaning performance). In this case, multipleobstructed states may be compared to the baseline ‘normal’ state. Themetric may be described in the notion of a normal state and the notionof interrupted or fail state based on the presence of an obstruction onthe lens of a camera. For example, a camera cleaning validation systemmay facilitate that a quality of a captured optical signal isindependent of the camera's resolution, field-of-view, orientation,lightning, or specific knowledge of the application of the camera'sdata. The camera cleaning validation system may facilitate a generalizedpass or fail criteria independent of the metric, under a degraded event,yet still be relevant to a broad set of applications (e.g., recognizingfaces, cars, etc.). The signal received from a degraded event using aparticular image (e.g., spilled coins image) is independent of thecamera's application but still represent correctness to a wide varietyof camera applications.

Therefore, a camera cleaning validation system would lend itself to abefore and after judgment and a notion of using a particular index thatwill allow a before and after score to whether a camera system isperforming to a predetermined level. The score may be tied to imagequality by using a particular index. In one or more embodiments, acamera cleaning validation system may facilitate a structural similarityindex measurement (SSIM) to be used as the particular index. The SSIMmay be a perceptual metric that quantifies image quality degradationcaused by processing an image. SSIM may provide a measure of theperceptual difference between two images. SSIM allows for a moregeneralized set of scores that apply more broadly to a large set ofimage applications. For example, traditional image quality metrics suchas slanted edges, color charts, or other metrics require an analysis ofdifferent metrics to get the full picture, whereas SSIM provides asimpler way to capture failure modes independent of the application.SSIM provides an index associated with a large amount of informationwhile using one score. A benefit to the SSIM metric is that it cancapture image degradation continuously across the whole image, whereasthe slanted edge and sharpness targets can only measure localizedchanges. Measuring uniform changes across the image is importantbecause 1) a localized obstruction may not land precisely on the slantedge or sharpness marker region and 2) it provides a broader picture ofthe uniformity of the obstruction. As long as a known good environmentis established, there is no need to calibrate a color chart or othermetrics before capturing an SSIM score. In this case, SSIM would providea pass or fail score which is a simple way to capture failureconditions. SSIM helps measure a target's noise structure, luminance,and other factors to understand the general image quality degradationbetween a dirty sensor and a clean sensor. SSIM also gives a generalscore for the entire target or a cropped region of the target.

In one or more embodiments, a camera cleaning validation system may usea particular pattern associated with the application-independentmethodology used to validate a camera system under test where anobstruction may have been deposited on a lens of a camera. Theparticular pattern may be used in conjunction with an SSIM scale in thebefore and after judgment. The particular pattern may comprise a varietyof textures and frequency of content that allows it to be applied to adifferent camera field of use, different camera distances, and becomesagnostic of the camera system set up compared to other approaches inperforming optical tests. The particular pattern may be a spilled coinsimage, which is typically used to measure a texture sharpness. A spilledcoins image is an image processing pattern that allows the evaluation ofsharpness. It has a frequency of content pattern which allows it tojudge that had a variety of extra detail levels. Other types ofsharpness targets do not capture that as well.

In one or more embodiments, a camera cleaning validation system mayfacilitate a novel linkage of applying a metric (e.g., SSIM) to a camerasystem under the introduction of an obstruction to the lens of a camera.The camera may be related to one of wide-field view cameras, stereocameras, automotive cameras, backup cameras, color or monochromecameras, autonomous vehicle cameras, or anything that utilizes anoptical path.

In one or more embodiments, a camera cleaning validation system mayfacilitate judging whether a processing circuitry of a vehicle wouldperceive an image captured by a camera of the vehicle to be what it issupposed to be when the camera is subjected to the obstruction. Thetesting and validation may be performed quickly and effortlesslyindependently from the judgment criteria that would have to be derivedfrom some image quality standard metric or some image qualityspecification, for example, a sharpness test.

In one or more embodiments, a camera cleaning validation system mayfacilitate the processing of an SSIM score over a plurality of samplesto determine regions of worst SSIM performance of the camera systembefore and after applying a camera cleaning system to attempt to returnthe camera to its normal function by clearing the obstruction off of thecamera lens. For example, regions with worst SSIM (e.g., worstperformance) based on a comparison to a validation threshold are boxedsuch that it may be determined that one area of the camera lens isperforming better than the boxed regions. A global SSIM may bedetermined based on capturing an image, down-sampling the image, takingSSIM scores of the down-sampled image, and repeating this processmultiple times. The global SSIM may be measured by looking at theaverage of the changes in each individual pixel. The global SSIM enablesthe SSIM to be evaluated in a localized region of the image. This iseffective at detecting small localized regions of obstructions (e.g.,droplets, splats, or globs of material) that are very significant butwould be removed from the global SSIM calculation. Downsampling can beachieved by taking a reference and distorted image signals as input, thesystem iteratively applies a low-pass filter and downsamples thefiltered image by a factor of 2. This may be performed at the cleanstate, the dirty state, and the post-cleaning state. A captured image isdown-sampled to remove noise resulting from factors such as distance,lighting, resolution, field-of-view, orientation, etc. Afterdown-sampling, SSIM scores are taken of the various regions of thedown-sampled image. This process may be repeated multiple times tominimize the effect of noise (e.g., obstruction). The global SSIM scoremay be determined based on averaging the SSIM scores associated witheach down-sampled image. This global SSIM score has the benefit of beingindependent of setup, quality of the lighting, distance, resolution,field-of-view, orientation, or other camera dependent conditions. Thedown-sampling may be performed using software.

In one or more embodiments, a camera cleaning validation system mayimpose a threshold to create the boxed regions associated with the worstSSIM scores. For example, any localized SSIM score that is greater thanthe threshold is associated with a lighter color on the a visualrepresentation of the localized SSIM scores and any SSIM score that islower than the threshold may be associated with a darker color on avisual representation of the localized SSIM score. Further, regions ofdarker colors that are connected may be represented by a box drawnaround those regions when the connected regions are larger than a givensize, for example, 10×10 pixels. Taking 10×10 pixels as an example,regions that are connected together but are smaller than 10×10 pixelsmay be considered as insignificant imperfection. It should be understoodthat darker regions on the global SSIM score have worse SSIM scores thanlighter regions.

In one or more embodiments, a camera cleaning validation system mayfacilitate that the system administrator may analyze the global SSIMscore to determine regions that are boxed and are below the threshold toconclude that the corresponding region of the camera lens has notproperly been cleaned using the camera cleaning system. This may alsoindicate that the cleaning system may need to be adjusted to addressthose regions in an attempt to restore the camera lens to its originaloperating state. This analysis may also indicate whether the cameracleaning system does not pass the operating threshold and may beconsidered to be defective. However, if the camera cleaning system wascapable of addressing the boxed region by clearing the obstruction fromthose regions having a low SSIM score, then the camera cleaning systemmay be considered to pass the operating threshold and is adequate.

The threshold above descriptions are for purposes of illustration andare not meant to be limiting. Numerous other examples, configurations,processes, etc., may exist, some of which are described in greaterdetail below. Example embodiments will now be described with referenceto the accompanying figures.

FIG. 1 illustrates an exemplary vehicle 100 equipped with multiplecameras. The vehicle 100 may be one of the various types of vehiclessuch as a gasoline-powered vehicle, an electric vehicle, a hybridelectric vehicle, or an autonomous vehicle, and can include variousitems such as a vehicle computer 105 and an auxiliary operationscomputer 110.

The vehicle computer 105 may perform various functions such ascontrolling engine operations (fuel injection, speed control, emissionscontrol, braking, etc.), managing climate controls (air conditioning,heating, etc.), activating airbags, and issuing warnings (check enginelight, bulb failure, low tire pressure, a vehicle in a blind spot,etc.).

The auxiliary operations computer 110 may be used to support variousoperations in accordance with the disclosure. In some cases, some or allof the components of the auxiliary operations computer 110 may beintegrated into the vehicle computer 105. Accordingly, variousoperations in accordance with the disclosure may be executed by theauxiliary operations computer 110 in an independent manner. For example,the auxiliary operations computer 110 may carry out some operationsassociated with providing camera settings of one or more cameras in thevehicle without interacting with the vehicle computer 105. The auxiliaryoperations computer 110 may carry out some other operations incooperation with the vehicle computer 105. For example, the auxiliaryoperations computer 110 may use information obtained by processing avideo feed from a camera to inform the vehicle computer 105 to execute avehicle operation such as braking.

In the illustration shown in FIG. 1 , the vehicle 100 is equipped withfive cameras. In other scenarios, fewer or a greater number of camerasmay be provided. The five cameras include a front-facing camera 115, arear-facing camera 135, a roof-mounted camera 130, a driver-side mirrorcamera 120, and a passenger-side mirror camera 125. The front-facingcamera 115, which may be mounted upon one of various parts in the frontof the vehicle 100, such as a grille or a bumper, produces images thatmay be used, for example, by the vehicle computer 105 and/or by theauxiliary operations computer 110, to interact with an automatic brakingsystem of the vehicle 100. The automatic braking system may slow downthe vehicle 100 if the images produced by the front-facing camera 115indicate that the vehicle 100 is too close to another vehicle travelingin front of the vehicle 100.

Any of the various cameras (e.g., cameras 115, 120, 125, 130, and 135)should not be interrupted from its normal function under the presence ofobstructions such as debris, mud, rain, bugs, or other obstructions thathinder the normal operation of the camera. Captured data by the cameras(e.g., cameras 115, 120, 125, 130, and 135) may be raw data that is sentto a vehicle computer 105 and/or by the auxiliary operations computer110 order to convert the optical image into processed signals.Therefore, it is desirable to enhance the testing and validation ofthese various cameras before real-world applications (e.g., being on theroad) to ensure that they do not provide inconsistent or unreliableimage quality that undermines their normal operation.

The rear-facing camera 135 may be used, for example, to display upon adisplay screen of an infotainment system 111, images of objects locatedbehind the vehicle 100. A driver of the vehicle 100 may view theseimages when performing a reversing operation upon the vehicle 100.

The roof-mounted camera 130 may be a part of an autonomous drivingsystem when the vehicle 100 is an autonomous vehicle. Images produced bythe roof-mounted camera 130 may be processed by the vehicle computer 105and/or by the auxiliary operations computer 110 for detecting andidentifying objects ahead and/or around the vehicle. The roof-mountedcamera 130 can have a wide-angle field-of-view and/or may be rotatableupon a mounting base. The vehicle 100 can use information obtained fromthe image processing to navigate around obstacles.

The driver-side mirror camera 120 may be used for capturing images ofvehicles in an adjacent lane on the driver side of the vehicle 100 andthe passenger-side mirror camera 125 may be used for capturing images ofvehicles in adjacent lanes on the passenger side of the vehicle 100. Inan exemplary application, various images captured by the driver-sidemirror camera 120, the passenger-side mirror camera 125, and therear-facing camera 135 may be combined by the vehicle computer 105and/or by the auxiliary operations computer 110 to produce acomputer-generated image that provides a 360-degree field-of-coveragearound the vehicle 100. The computer-generated image may be displayedupon a display screen of the infotainment system 111 to assist thedriver drive the vehicle 100.

The various cameras provided in the vehicle 100 can be any of varioustypes of cameras and can incorporate various types of technologies. Forexample, a night-vision camera having infra-red lighting and sensors maybe used for capturing images in low light conditions. The low lightconditions may be present, for example, when the vehicle 100 is parkedat a spot during the night. The images captured by the night-visioncamera may be used for security purposes such as for preventingvandalism or theft. A stereo camera may be used to capture images thatprovide depth information that may be useful for determining separationdistance between the vehicle 100 and other vehicles when the vehicle 100is in motion. In another application where minimal processing latency isdesired, a pair of cameras may be configured for generating a highframe-rate video feed. The high frame-rate video feed may be generatedby interlacing the video feeds of the two cameras. In yet anotherapplication, a camera system configured for light detection and ranging(LIDAR) applications may be used. LIDAR applications can includelong-distance imaging and/or short distance imaging. Some camera systemsmay include power-saving features that may be useful for operations incertain environments.

In one or more embodiments, a camera cleaning validation system mayfacilitate the setup of a camera (e.g., cameras 115, 120, 125, 130, or135) in a test environment which may be constrained in both its requiredsetup as well as the environment it is in. Cameras (e.g., cameras 115,120, 125, 130, and 135) may be subjected to obstructions before beingintroduced in real-world scenarios where the cameras need to operate atan optimal level to ensure quality images are being captured andprocessed with minimal errors. A camera (e.g., cameras 115, 120, 125,130, or 135) may be interrupted from its normal function under thepresence of an obstruction, which would alter the image quality capturedby the camera. For example, obstructions may include debris, mud, rain,bugs, or other obstructions that hinder the normal operation of thecamera. These obstructions may cause interference and alteration of theimage quality of a camera. It is important to note that an obstructioncan reduce the image quality in any combination of a uniform obstructionor a single or series of localized obstructions.

In one or more embodiments, a camera cleaning validation system mayprovide a mechanism to allow a pass or fail criteria to be judged on acamera (e.g., cameras 115, 120, 125, 130, or 135) under test inreal-time during the testing and provides a target framework and abackend processing framework together because of the real-timeapplication. A camera cleaning validation system may allow for a testingand validation environment that does not require having to tune thecamera (e.g., cameras 115, 120, 125, 130, or 135) for ideal lightningconditions, setup, distance, etc., which are typically required bycamera tests. That is, there is no need to determine pixel sizes,determine a specific color palette, which is typically used to performtests.

In one or more embodiments, a camera cleaning validation system mayfacilitate a structural similarity index measurement (SSIM) to be usedas the particular index. It should be noted that other options may alsobe used as the particular index based on the camera application. Othermetrics or targets could be used instead of SSIM. For example, asharpness or texture or signal-to-noise metric may be used instead ofSSIM. An important aspect of the camera cleaning validation system isthat it facilitates 1) comparing the clean (baseline) and post-clean(sample) images and 2) evaluating both global and localizedperformances. The SSIM may be a perceptual metric that quantifies imagequality degradation caused by processing an image. SSIM may provide ameasure of the perceptual difference between two images.

In one or more embodiments, a camera cleaning validation system may usea particular pattern associated with the application-independentmethodology used to validate a camera system under test where anobstruction may have been deposited on a lens of a camera. Theparticular pattern may be used in conjunction with an SSIM scale in thebefore and after judgment. The particular pattern may comprise a varietyof textures and frequency of content that allows it to be applied to adifferent camera field of use, different camera distances, and becomesagnostic of the camera system set up compared to other approaches inperforming optical tests. The particular pattern may be a spilled coinsimage, which is typically used to measure a texture sharpness. A spilledcoins image is an image processing pattern that allows the evaluation ofsharpness. It has a frequency of content pattern which allows it tojudge that had a variety of extra detail levels. Other types ofsharpness targets do not capture that as well.

It is understood that the above descriptions are for purposes ofillustration and are not meant to be limiting.

FIG. 2 depicts an illustrative schematic diagram for testing andvalidation, in accordance with one or more example embodiments of thepresent disclosure.

Referring to FIG. 2 , there is shown a camera cleaning validation system200 for verifying the status of a camera 202. The camera cleaningvalidation system 200 may be connected to a computer 201. The computer201 may provide a system administrator access to inputs and outputs ofthe camera cleaning validation system 200. The computer 201 may alsocontrol the camera cleaning validation system 200 by adjustingparameters associated with the various components of the camera cleaningvalidation system 200. Camera 202 or any other cameras discussed in thefollowing figures may be any of the cameras depicted and discussed inFIG. 1 . The camera may include a variety of cameras such as wide-fieldview cameras, stereo cameras, backup cameras, color or monochromecameras, autonomous vehicle cameras, or any camera that utilizes anoptical path. The camera cleaning validation system 200 comprises thecamera 202, a power source 204, and object 206, a camera cleaning system207, and an obstruction source 208. The camera 202 may be positioned infront of the object 206 to capture one or more images of the object 206under normal conditions and other conditions. A normal condition may beconsidered a condition where noise, interference, or other imagedegrading conditions are not introduced to the camera 202. Under itsnormal condition, the camera 202 captures an optical image as input datafrom the object. The captured input data may be raw data that is sent toa processing unit associated with the camera device to convert theoptical image into processed signals. The camera 202 may be connected toa scoring module 210 that provides camera verification by making ajudgment about the raw data to determine if the camera passes avalidation threshold. The scoring module 210 may utilize a structuralsimilarity index measurement (SSIM) as a scoring index. The SSIM scoringindex may be a perceptual metric that quantifies image qualitydegradation caused by processing an image. SSIM may provide a measure ofthe perceptual difference between a before and after images. The camera202 may capture a plurality of images of object 206 and uses the scoringmodule 210 to validate each image quality using SSIM. The scoring module210 may record the values registered based on SSIM. In one or moreembodiments, the object 206 may be a spilled coins image, which istypically used to measure a texture sharpness. A spilled coins image isan image processing pattern that allows the evaluation of sharpness. Ithas a frequency of content pattern which allows it to judge that had avariety of extra detail levels. Other types of sharpness targets do notcapture that as well. In one or more embodiments, the obstruction source208 may introduce the camera 202 to various intensities and differentfrequencies of radiant energy. The scoring module 210 may provide scoresassociated with the image qualities of images being captured under theinfluence of the obstruction source 208.

The scoring module 210 may evaluate a captured image of the object 206to determine SSIM scores of different regions of the captured image. Forexample, SSIM scores may be assigned to different regions of thecaptured image. These SSIM scores would then be compared to apredetermined threshold (e.g., an SSIM score of approximately 0.9).Based on that comparison the scoring module 210 may make a judgment towhether the normal function of the camera 202 has been disrupted basedon the SSIM scores resulting from the camera 202 capturing images whilebeing subjected to the obstruction introduced by the obstruction source208. For example, an image may be first captured by camera 202 undernormal conditions (e.g., without influence from the obstruction source208), which may be referred to as a clean state. This image may bescored by the scoring module 210 using the SSIM scoring index todetermine a baseline of SSIM scores in the clean state.

In one or more embodiments, a camera cleaning validation system maycapture an image using the camera 202 after an obstruction is applied tothe camera lens using the obstruction source 208. This state may beconsidered as a dirty state. The image may be associated with anobstruction level that has been introduced to the camera 202 using theobstruction source 208. It may be important to capture the SSIM scoresassociated with the dirty lens before applying the camera cleaningsystem 207 in order to ensure consistency between tests. That is, havinga consistent obstruction level introduced on the lens of camera 202ensures the validation of a camera cleaning system 207 is consistentover a plurality of tests.

The camera cleaning system 207 may be applied after the camera cleaningvalidation system captures the image under the dirty state. The cameracleaning system 207 may apply fluids through a nozzle or airflow to thelens in an attempt to remove the obstruction introduced by theobstruction source 208. The application of fluids or airflow may becontrolled by the camera cleaning system 207 in order to vary theconcentration and pressure of fluids, the speed of the airflow, and/orthe angle of the fluid nozzle or the airflow nozzle. In addition, thedirection of fluids and airflow may be also controlled by the cameracleaning system 207. At this point, the state of the camera cleaningvalidation system may be considered a post-cleaning state.

In one or more embodiments, a camera cleaning validation system maycapture an image of the object 206 after the application of the cameracleaning system 207. The scoring module 210 may evaluate the image todetermine SSIM scores of different regions on the image. For example,SSIM scores may be assigned to different regions of the captured image.These SSIM scores may then be compared to a predetermined threshold(e.g., an SSIM score of approximately 0.9).

In one or more embodiments, a camera cleaning validation system maydetermine whether the camera's operation has been disrupted to a pointto classify the camera cleaning system 207 to be in a failed state. Forexample, the SSIM scores in the post-clean state may be compared to avalidation threshold. In case the SSIM scores are below the validationthreshold, the camera cleaning system 207 may be considered to be in afailed state. However, if the SSIM scores are above the validationthreshold, the camera cleaning system 207 may be considered to be in apassing state.

It is understood that the above descriptions are for purposes ofillustration and are not meant to be limiting.

FIG. 3A depicts an illustrative schematic diagram 300 for testing andvalidation of a camera cleaning system, in accordance with one or moreexample embodiments of the present disclosure.

Referring to FIG. 3A, there is shown a camera 302 pointed to a spilledcoins image 304. The camera 302 may be connected to a scoring module306. The spilled coins image 304 comprises a variety of contentfrequencies, which makes it suitable for capturing degradations due toenvironmental interference, such as debris, mud, rain, bugs, or otherobstructions that hinder the normal operation of a camera. FIG. 3Arepresents a normal condition (e.g., a clean state) under which camera302 is operating. This normal condition, where a minimal to no amount ofobstruction may be present, may represent a baseline to be used whencomparing images captured of the spilled coins image 304 when camera 302is subjected to an obstruction.

In FIG. 3A, the spilled coins image 304 may comprise a plurality ofregions (e.g., regions 301, 303, 305, etc.), where each of these regionscontains content that varies from another region. For example, images ofregion 301, region 303, and region 305 may result in respectivebaselines for each of those regions. After an image is captured with thecamera 302, a comparison between the baseline image and the capturedimage may result in a variety of SSIM scores associated with thoseregions. That is, region 301 may have a different SSIM score than region303 and region 305. In some embodiments, each of these SSIM scores maybe compared to a validation threshold to determine a status of thecamera 302. For example, the scoring module 306 may generate SSIM scoresfor each of the regions 301, 303, and 305 and then compare each of therespective SSIM scores to a validation threshold to determine whetherany of these regions are below the validation threshold, which wouldindicate that the camera 302 may be judged to be in a failed state. Forexample, if a single region SSIM score is below the validation threshold(e.g., 0.9), then it may be determined that the camera is in a failedstate because there is no obstruction applied to the camera, yet it isshowing SSIM scores lower than the validation threshold. In some otherexamples, it may be determined that a certain number of regions SSIMscores have to be below the validation threshold before determining thatthe camera has failed. Considering that FIG. 3A represents the normalcondition under which camera 302 is operating, the SSIM scoresassociated with the various regions of the spilled coins image 304 maybe expected to be above the validation threshold. SSIM is a method forpredicting the perceived quality of digital television and cinematicpictures, as well as other kinds of digital images and videos. SSIM isused for measuring the similarity between two images. The SSIM index isa full reference metric. In other words, the measurement or predictionof image quality is based on an initial image or distortion-free imageas a reference. SSIM scores may be generated between the normalcondition (clean state) of camera images and images taken after thecamera 302 is subjected to an obstruction (e.g., a dirty state). Thesescores are then compared to a validation threshold to determine whetherthe SSIM scores are below or above the validation threshold.

It is understood that the above descriptions are for purposes ofillustration and are not meant to be limiting.

FIG. 3B depicts an illustrative schematic diagram 350 for testing andvalidation of a camera cleaning system, in accordance with one or moreexample embodiments of the present disclosure.

FIG. 3B depicts a resulting image 304′ where the camera 302 captured animage of the spilled coins 304 of FIG. 3A after the application of anobstruction using the obstruction source 308. In FIG. 3A, the testingand validation may start with a camera 302 that is known to operateunder normal conditions and determining an expected response tocapturing images of a known target (e.g., a spilled coins image). InFIG. 3B, the normal conditions are interrupted by subjecting the lens ofcamera 302 to an obstruction using the obstruction source 308. Theobstruction may include debris, mud, rain, bugs, or other obstructionsthat hinder the normal operation of the camera 302. This state may beconsidered as a dirty state. An image may be captured using apredetermined obstruction level introduced to the camera 302 using theobstruction source 308. It is important to capture SSIM scoresassociated with the dirty lens before applying the camera cleaningsystem 307 in order to ensure consistency between tests. That is, havinga consistent obstruction level introduced on the lens of camera 302ensures the validation of a camera cleaning system 307 is consistentover a plurality of tests.

In the environment of FIG. 3B, as camera 302 is subjected to anobstruction using the obstruction source 308, the resulting imagecaptured by camera 302 may be degraded due to the obstruction. Thecamera 302 may be connected to the scoring module 306. As explainedabove, the spilled coins image 304 of FIG. 3A comprises a variety ofcontent frequencies, which makes it suitable for capturing degradationsdue to obstructions. While camera 302 is subjected to obstructions,camera 302 may capture optical signals as raw data associated with thevariety of regions included in spilled coins image 304 of FIG. 3A. Thecamera 302 may send the raw data to a processing unit associated withthe camera 302 to convert the optical images into processed signals. Aswas shown in FIG. 3A, the spilled coins image 304 comprised a pluralityof regions (e.g., regions 301, 303, 305, etc.). FIG. 3B shows adistorted image 304′ due to obstructions introduced by the obstructionsource 308. Images may be represented by raw data captured while camera302 is subjected to obstructions during the capture of optical signalsfrom the spilled coins image 304 in the same regions as represented inregions 301′, 303′, and 305′ which correspond to regions 301, 303, and305 of FIG. 3A, respectively. Images of region 301′, region 303′, andregion 305′ may result in respective image changes for each of thoseregions due to the obstructions. The scoring module 306 may determineSSIM scores associated with each of those regions by analyzing the rawdata, independent of the application of the distorted image 304′. Thatis, region 301′ may have a different SSIM score than region 303′ andregion 305′.

The camera cleaning system 307 may be applied after capturing the imageunder the dirty state. The camera cleaning system 307 may apply fluidsthrough a nozzle or airflow to the lens in an attempt to remove theobstruction introduced by the obstruction source 308. The application offluids or airflow may be controlled by the camera cleaning system 307 inorder to vary the concentration and pressure of fluids and the speed ofthe airflow. In addition, the direction of fluids and airflow may bealso controlled by the camera cleaning system 307. At this point, thestate of camera 302 may be considered a post-cleaning state. The camera302 may capture an image of the spilled coins image 304 after theapplication of the camera cleaning system 307. The scoring module 306may evaluate the image to determine SSIM scores of different regions onthe captured image. For example, SSIM scores may be assigned todifferent regions of the captured image. These SSIM scores may then becompared to a predetermined threshold (e.g., an SSIM score ofapproximately 0.9).

In one or more embodiments, a camera cleaning validation system maydetermine whether the camera's operation has been disrupted to a pointto classify the camera cleaning system 307 to be in a failed state. Forexample, the SSIM scores in the post-clean state may be compared to avalidation threshold. In case the SSIM scores are below the validationthreshold, the camera cleaning system 207 may be considered to be in afailed state. However, if the SSIM scores are above the validationthreshold, the camera cleaning system 207 may be considered to be in apassing state.

FIG. 3C depicts an illustrative schematic diagram 380 for testing andvalidation a camera cleaning system, in accordance with one or moreexample embodiments of the present disclosure.

Referring to FIG. 3C, there is shown images 310 and 312 show one or moreareas that result in a variety of global scores. For example, image 310shows the various regions shown with light to dark variation of colorscorresponding to an image taken after the obstruction was introduced tothe lens of the camera 302.

In one or more embodiments, a camera cleaning validation system mayfacilitate capturing SSIM scores over a plurality of tests to determineregions of worst SSIM performance of the camera system before and afterapplying the camera cleaning system 307 to attempt to return the camerato its normal function by clearing the obstruction off of the cameralens.

For example, regions with the worst SSIM (e.g., worst performance) basedon a comparison to a validation threshold are represented as boxed areas(e.g., boxed areas 320, 321, and 322, note that not all boxed areas arelabeled). A global SSIM may be determined based on capturing an image,down-sampling the image, taking SSIM scores of the down-sampled image,and repeating this process multiple times. A captured image isdown-sampled to remove noise resulting from factors such as distance,lighting, resolution, field-of-view, orientation, etc. Afterdown-sampling, SSIM scores are taken of the various regions of thedown-sampled image. This process may be repeated multiple times tominimize the effect of noise. The global SSIM score may be determinedbased on averaging the SSIM scores associated with each down-sampledimage. This global SSIM score has the benefit of being independent ofsetup, quality of the lighting, distance, resolution, field-of-view,orientation, or other camera dependent conditions. Therefore, image 310shows a resulting image taken of a spilled coins image after the lens ofthe camera has been subjected to an obstruction. Also, the resultingimage is a result of the plurality of image down-sampling and capturingthe SSIM scores at each iteration, which are then averaged to generateglobal SSIM scores of the various regions of the image 310.

In one or more embodiments, a camera cleaning validation system mayimpose a threshold to create the boxed regions associated with the worstSSIM scores. For example, any SSIM score of image 310 that is greaterthan the threshold is associated with a lighter color on the global SSIMscore and any SSIM score of image 310 that is lower than the thresholdmay be associated with a darker color on the global SSIM score. Itshould be understood that darker regions on image 310 represent globalSSIM scores that have worse SSIM scores than lighter regions.

In one or more embodiments, the scoring module 306 may convert image 310into image 312 by determining that regions of darker colors that areconnected together are represented by a box drawn around those regionswhen the connected regions are larger than a given size, for example,10×10 pixels. Taking 10×10 pixels as an example, regions that areconnected together but are smaller than 10×10 pixels may be consideredas insignificant imperfection. It should be understood that 10×10 pixelsis an example and should not be limiting. Other region sizes may beselected for consideration.

In one or more embodiments, a camera cleaning validation system mayfacilitate that the system administrator may analyze the global SSIMscores and the boxed regions to determine that they are below thethreshold. The system administrator may conclude that the correspondingregion of the camera lens has not been properly cleaned using the cameracleaning system 307 of FIG. 3A. This may also indicate either that thefluid nozzle pressure or the airflow speed may need to be adjusted toaddress those regions in an attempt to restore the camera lens to itsoriginal operating state. This analysis may also indicate whether thecamera cleaning system 307 does not pass the operating threshold and maybe considered to be defective. However, if the camera cleaning system307 was capable of correcting the boxed region by clearing theobstruction from those regions having a low SSIM score, then the cameracleaning system may be considered to pass the operating threshold andmay be considered to be in an operational state. It is understood thatthe above descriptions are for purposes of illustration and are notmeant to be limiting.

FIG. 4 illustrates a flow diagram of illustrative process 400 for acamera cleaning validation system, in accordance with one or moreexample embodiments of the present disclosure.

At block 402, a device may capture a first image of a target using acamera, wherein the camera is in a clean state, and wherein the targetis in a line of sight of the camera.

At block 404, the device may apply an obstruction to a portion of a lensof the camera.

At block 406, the device may apply a camera cleaning system to the lensof the camera.

At block 408, the device may capture a post-clean image after applyingthe camera cleaning system.

At block 410, the device may determine a post-clean SSIM score based oncomparing the post clean image to the first image. The device maydetermine a first down-sampled image based on the down-sampling thepost-clean image. The device may determine a first local SSIM scorebased on comparing the first image and the post-clean down-sampledimage. The device may determine a second down-sampled image based on thedown-sampling of the first down-sampled image. The device may determinea second local SSIM score based on comparing the first down-sampledimage and the second down-sampled image. The device may determine aglobal SSIM score, wherein the global SSIM score is an average of thefirst local SSIM score and the second local SSIM score. The device maydetermine a first region of the post-clean image having SSIM scoresbelow a validation threshold. The device may determine a second regionof the post-clean image having SSIM scores below the validationthreshold. The device may determine the first region and the secondregion are connected together. The device may display a box around thefirst region and the second region. The device may flag the displayedbox as a degraded area that has a worse SSIM score than an area withouta box around it.

At block 412, the device may compare the post-clean SSIM score to avalidation threshold.

At block 414, the device may determine a validation state of the cameracleaning system based on the comparison. The device may perform one ormore adjustments to the camera cleaning system to mitigate the degradedarea.

It is understood that the above descriptions are for purposes ofillustration and are not meant to be limiting.

FIG. 5 is a block diagram illustrating an example of a computing deviceor computer system 500 upon which any of one or more techniques (e.g.,methods) may be performed, in accordance with one or more exampleembodiments of the present disclosure.

For example, the computing system 500 of FIG. 5 may represent thevehicle computer 105 and/or the auxiliary operations computer 110 ofFIG. 1 and/or the computer 201 of FIG. 2 . The computer system (system)includes one or more processors 502-506. Processors 502-506 may includeone or more internal levels of cache (not shown) and a bus controller(e.g., bus controller 522) or bus interface (e.g., I/O interface 520)unit to direct interaction with the processor bus 512. A camera cleaningvalidation device 509 may also be in communication with the Processors502-506 and may be connected to the processor bus 512.

Processor bus 512, also known as the host bus or the front side bus, maybe used to couple the processors 502-506 and/or the camera cleaningvalidation device 509 with the system interface 524. System interface524 may be connected to the processor bus 512 to interface othercomponents of the system 500 with the processor bus 512. For example,system interface 524 may include a memory controller 518 for interfacinga main memory 516 with the processor bus 512. The main memory 516typically includes one or more memory cards and a control circuit (notshown). System interface 524 may also include an input/output (I/O)interface 520 to interface one or more I/O bridges 525 or I/O devices530 with the processor bus 512. One or more I/O controllers and/or I/Odevices may be connected with the I/O bus 526, such as I/O controller528 and I/O device 530, as illustrated.

I/O device 530 may also include an input device (not shown), such as analphanumeric input device, including alphanumeric and other keys forcommunicating information and/or command selections to the processors502-506 and/or the camera cleaning validation device 509. Another typeof user input device includes cursor control, such as a mouse, atrackball, or cursor direction keys for communicating directioninformation and command selections to the processors 502-506 and/or thecamera cleaning validation device 509 and for controlling cursormovement on the display device.

System 500 may include a dynamic storage device, referred to as mainmemory 516, or a random access memory (RAM) or other computer-readabledevices coupled to the processor bus 512 for storing information andinstructions to be executed by the processors 502-506 and/or the mycamera cleaning validation device 509. Main memory 516 also may be usedfor storing temporary variables or other intermediate information duringexecution of instructions by the processors 502-506 and/or the cameracleaning validation device 509. System 500 may include read-only memory(ROM) and/or other static storage device coupled to the processor bus512 for storing static information and instructions for the processors502-506 and/or the camera cleaning validation device 509. The systemoutlined in FIG. 5 is but one possible example of a computer system thatmay employ or be configured in accordance with aspects of the presentdisclosure.

According to one embodiment, the above techniques may be performed bycomputer system 500 in response to processor 504 executing one or moresequences of one or more instructions contained in main memory 516.These instructions may be read into main memory 516 from anothermachine-readable medium, such as a storage device. Execution of thesequences of instructions contained in main memory 516 may causeprocessors 502-506 and/or the camera cleaning validation device 509 toperform the process steps described herein. In alternative embodiments,circuitry may be used in place of or in combination with the softwareinstructions. Thus, embodiments of the present disclosure may includeboth hardware and software components.

Various embodiments may be implemented fully or partially in softwareand/or firmware. This software and/or firmware may take the form ofinstructions contained in or on a non-transitory computer-readablestorage medium. Those instructions may then be read and executed by oneor more processors to enable the performance of the operations describedherein. The instructions may be in any suitable form, such as, but notlimited to, source code, compiled code, interpreted code, executablecode, static code, dynamic code, and the like. Such a computer-readablemedium may include any tangible non-transitory medium for storinginformation in a form readable by one or more computers, such as but notlimited to read-only memory (ROM); random access memory (RAM); magneticdisk storage media; optical storage media; a flash memory, etc.

A machine-readable medium includes any mechanism for storing ortransmitting information in a form (e.g., software, processingapplication) readable by a machine (e.g., a computer). Such media maytake the form of, but is not limited to, non-volatile media and volatilemedia and may include removable data storage media, non-removable datastorage media, and/or external storage devices made available via awired or wireless network architecture with such computer programproducts, including one or more database management products, web serverproducts, application server products, and/or other additional softwarecomponents. Examples of removable data storage media include CompactDisc Read-Only Memory (CD-ROM), Digital Versatile Disc Read-Only Memory(DVD-ROM), magneto-optical disks, flash drives, and the like. Examplesof non-removable data storage media include internal magnetic harddisks, SSDs, and the like. The one or more memory devices 606 (notshown) may include volatile memory (e.g., dynamic random access memory(DRAM), static random access memory (SRAM), etc.) and/or non-volatilememory (e.g., read-only memory (ROM), flash memory, etc.).

Computer program products containing mechanisms to effectuate thesystems and methods in accordance with the presently describedtechnology may reside in main memory 516, which may be referred to asmachine-readable media. It will be appreciated that machine-readablemedia may include any tangible non-transitory medium that is capable ofstoring or encoding instructions to perform any one or more of theoperations of the present disclosure for execution by a machine or thatis capable of storing or encoding data structures and/or modulesutilized by or associated with such instructions. Machine-readable mediamay include a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more executable instructions or data structures.

Embodiments of the present disclosure include various steps, which aredescribed in this specification. The steps may be performed by hardwarecomponents or may be embodied in machine-executable instructions, whichmay be used to cause a general-purpose or special-purpose processorprogrammed with the instructions to perform the steps. Alternatively,the steps may be performed by a combination of hardware, software,and/or firmware.

Various modifications and additions can be made to the exemplaryembodiments discussed without departing from the scope of the presentinvention. For example, while the embodiments described above refer toparticular features, the scope of this invention also includesembodiments having different combinations of features and embodimentsthat do not include all of the described features. Accordingly, thescope of the present invention is intended to embrace all suchalternatives, modifications, and variations together with allequivalents thereof.

The operations and processes described and shown above may be carriedout or performed in any suitable order as desired in variousimplementations. Additionally, in certain implementations, at least aportion of the operations may be carried out in parallel. Furthermore,in certain implementations, less than or more than the operationsdescribed may be performed.

The word “exemplary” is used herein to mean “serving as an example,instance, or illustration.” Any embodiment described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other embodiments.

As used herein, unless otherwise specified, the use of the ordinaladjectives “first,” “second,” “third,” etc., to describe a commonobject, merely indicates that different instances of like objects arebeing referred to and are not intended to imply that the objects sodescribed must be in a given sequence, either temporally, spatially, inranking, or any other manner.

It is understood that the above descriptions are for purposes ofillustration and are not meant to be limiting.

Although specific embodiments of the disclosure have been described, oneof ordinary skill in the art will recognize that numerous othermodifications and alternative embodiments are within the scope of thedisclosure. For example, any of the functionality and/or processingcapabilities described with respect to a particular device or componentmay be performed by any other device or component. Further, whilevarious illustrative implementations and architectures have beendescribed in accordance with embodiments of the disclosure, one ofordinary skill in the art will appreciate that numerous othermodifications to the illustrative implementations and architecturesdescribed herein are also within the scope of this disclosure.

Although embodiments have been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the disclosure is not necessarily limited to the specific featuresor acts described. Rather, the specific features and acts are disclosedas illustrative forms of implementing the embodiments. Conditionallanguage, such as, among others, “can,” “could,” “might,” or “may,”unless specifically stated otherwise, or otherwise understood within thecontext as used, is generally intended to convey that certainembodiments could include, while other embodiments do not include,certain features, elements, and/or steps. Thus, such conditionallanguage is not generally intended to imply that features, elements,and/or steps are in any way required for one or more embodiments or thatone or more embodiments necessarily include logic for deciding, with orwithout user input or prompting, whether these features, elements,and/or steps are included or are to be performed in any particularembodiment.

1. (canceled)
 2. A camera cleaning validation system comprising: acamera with a lens, the camera being configured to capture at least oneimage of a target; and a processor configured to: cause the camera tocapture a baseline image in response to the lens being in a clean state,control a cleaning system to initiate a cleaning operation to remove adetected obstruction from the lens, cause the camera to capture a postclean image after the cleaning operation, generate an image qualityscore based on a comparison between the baseline image and the postclean image, determine an operational state of the cleaning system inresponse to the image quality score being less than a validationthreshold, and adjust at least one parameter of the cleaning system inresponse to the operational state corresponding to a non-performingstate.
 3. The camera cleaning validation system of claim 2, wherein theimage quality score comprises a structural similarity index measurement(SSIM) score.
 4. The camera cleaning validation system of claim 3,wherein the processor is further configured to: perform a down samplingoperation of the post clean image to generate a first down sampledimage, and determine a first SSIM score based on comparing the baselineimage and the first down sampled image, the first SSIM score being afirst local SSIM score.
 5. The camera cleaning validation system ofclaim 4, wherein the processor is further configured to: perform asecond down sampling operation of the first down sampled image togenerate a second down sampled image; and determine a second SSIM scorebased on comparing the first down sampled image and the second downsampled image, the second SSIM score being a second local SSIM score. 6.The camera cleaning validation system of claim 5, wherein the processoris further configured to: generate a global SSIM score representing anaverage of the first local SSIM score and the second local SSIM score.7. The camera cleaning validation system of claim 6, wherein theprocessor is further configured to: identify, based on the global SSIMscore, regions of interest in the post clean image, the regions ofinterest corresponding to locations on the lens that have not beenproperly cleaned.
 8. The camera cleaning validation system of claim 3,wherein the processor is further configured to: identify a first regionof the post clean image having SSIM scores below the validationthreshold; identify a second region of the post clean image having SSIMscores below the validation threshold; and determine that the firstregion and the second region are connected together.
 9. The cameracleaning validation system of claim 8, wherein the processor is furtherconfigured to: display a bounding box around the first region, aroundthe second region, or around the first region and the second region inresponse to the first region, the second region, or a combined size ofthe first region and second region spanning an area greater than apredetermined pixel size.
 10. The camera cleaning validation system ofclaim 9, wherein the processor is further configured to: classify thedisplayed bounding box as a degraded area that has a worse SSIM scorethan an area without a box around it; and adjust the at least oneparameter of the cleaning system to mitigate the degraded area.
 11. Thecamera cleaning validation system of claim 2, wherein the at least oneparameter of the cleaning system comprises an airflow speed of thecleaning system.
 12. A sensor cleaning validation system comprising: aprocessor configured to: control a cleaning system to initiate acleaning operation to remove an obstruction from a lens of a sensor,cause the sensor to capture a post clean image after the cleaningoperation, generate an image quality score based on a comparison betweena baseline image and the post clean image, the baseline imagecorresponding to an image captured when the lens is determined to be ina clean state, determine an operational state of the cleaning system inresponse to the image quality score being less than a validationthreshold, and adjust at least one parameter of the cleaning system inresponse to the operational state corresponding to a non-performingstate.
 13. A sensor cleaning validation method comprising: causing, by aprocessor, a camera to capture a baseline image in response to a lens ofthe camera being in a clean state; controlling, by the processor, acleaning system to initiate a cleaning operation to remove a detectedobstruction from the lens; causing, by the processor, the camera tocapture a post clean image after the cleaning operation; generating, bythe processor, an image quality score based on comparing the baselineimage and the post clean image; determining, by the processor, anoperational state of the cleaning system based on comparing the imagequality score and a validation threshold; and adjusting, by theprocessor, at least one parameter of the cleaning system in response tothe operational state corresponding to a non-performing state.
 14. Themethod of claim 13, wherein the image quality score comprises astructural similarity index measurement (SSIM) score, the method furthercomprising: performing a down sampling operation of the post clean imageto generate a first down sampled image; and determining a first SSIMscore based on comparing the baseline image and the first down sampledimage, the first SSIM score being a first local SSIM score.
 15. Themethod of claim 14 further comprising: performing a second down samplingoperation of the first down sampled image to generate a second downsampled image; and determining a second SSIM score based on comparingthe first down sampled image and the second down sampled image, thesecond SSIM score being a second local SSIM score.
 16. The method ofclaim 15 further comprising: generating a global SSIM score representingan average of the first local SSIM score and the second local SSIMscore.
 17. The method of claim 16 further comprising: identifying, basedon the global SSIM score, regions of interest in the post clean image,the regions of interest corresponding to locations on the lens that havenot been properly cleaned.
 18. The method of claim 14 furthercomprising: identifying a first region of the post clean image havingSSIM scores below the validation threshold; identifying a second regionof the post clean image having SSIM scores below the validationthreshold; and determining that the first region and the second regionare connected together.
 19. The method of claim 18 further comprising:displaying a bounding box around the first region, around the secondregion, or around the first region and the second region in response tothe first region, the second region, or a combined size of the firstregion and second region spanning an area greater than a predeterminedpixel size.
 20. The method of claim 19 further comprising: classifyingthe displayed bounding box as a degraded area that has a worse SSIMscore than an area without a box around it; and adjusting the at leastone parameter of the cleaning system to mitigate the degraded area. 21.The method of claim 13, wherein the at least one parameter of thecleaning system comprises at least one of a fluid concentration and afluid pressure of the cleaning system.